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ABSTRACT 

A job evaluation system was designed and implemented 
by a college's personnel department using a "point dash factor" 
method for evaluating jobs. Employees made the major decision about 
which criteria should be used to rate job difficulty and how much 
weight each criterion should carry. Employee input was solicited 
through committees formed around occupations, a resolution committee 
comprised of representatives of the occupational committees, and a 
general mail survey of employees. After committees had formulated 
criteria, the criteria were further evaluated for bias, usefulness, 
and validity. The selected criteria were assigned to nine clusters 
and then submitted to the employees for weighting. Ratings assigned 
by the employees were then subjected to further reliability and 
validity tests including the measure of correlation to current salary 
grade. Results of the job evaluation procedure were then used to 
determine if pay inequities existed between female-dominated jobs and 
male-dominated jobs of similar difficulty. This was necessary in 
order to comply with state law. Although many female-dominated jobs 
were underpaid, others were not. Some male-dominated jobs were also 
not paid equitably. Salary adjustments will be made according to 
these guidelines with a supplementary procedure being used for 
documented recurring problems due to new lower salary ranges. The new 
point system will be used for classifying new jobs and reclassifying 
old ones. (ABL) 
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I . HISTORY OF THE PROJECT 



Job evaluation 1s a method for determining the difficulty of various 
classes of work based on the amount of skill, effort, and responsibility 
they each require. It provides a basis for assigning classes of work to 
different salary grades according to their relative difficulty. It also 
provides a basis for assigning jobs tc different classes of work according 
to the difficulty of the jobs. 

Concerns about the adequacy of the University's existing job evaluation 
methods prompted the Personnel Department to Initiate a research project that 
would evaluate alternative job evaluation systems and either choose or design 
one that would better meet the needs of the University. A review of the 
relevant literature Indicated that the University should replace Its three 
existing systems with a single one based on the "point-factor" method of 
evaluating jobs. This method rates jobs and classes of work according to 
fixed and explicit criteria of difficulty. Since deciding which criteria 
should be used 1s ultimately a value Judgment, the project focused on 
designing a new point- factor job evaluation system that Incorporated the 
criteria of University employees whose jobs would be affected. The process 
of developing, validating, and Implementing this job evaluation system is 
described below. 



) I . DEVELOPMENT OF THE JOB EVALUATION SYSTEM 

Employees made the major decisions about which criteria should be used 
to rate job difficulty and how much weight each criterion should carry. 
A test phase suggested modifications that would Improve the technical quality 
of the system. The final job evaluation system that emerged from the test 
phase proved to be very reliable for rating Individual jobs. It also rated 
the relative difficulty of classifications 1n a way that was consistent 
with their current salary relationships, with the judgments of experts 1n the 
University's Personnel Department, and with the standards of job difficulty 
Included 1n two job evaluation systems that are widely used throughout the 
country. 



Employee Input . The criteria were chosen by two types of employee 
committees: (a) 18 committees with seven to eight members each that were 
formed along occupational lines, and (b) a resolution committee composed 
of one member from each of the occupational committees. A general survey 
mailed to 8,000 non-hospital employees determined how the final criteria 
should be weighted. 

The occupational committees were formed by 138 employees who were 
randomly selected to be representative of the diversity, size, and sex 
composition of approximately 450 non -hospital classes of work at the Univer- 
sity. As a result of this procedure, 36% of the committee members repre- 
sented clerical and secretarial classes, 18% represented scientific classes, 
and the remaining 46% represented classes that dealt with administration, 
support services, physical operations, financial services, data processing, 
social and health services, art, Information, and student personnel services. 
Overall, 73* of the comnittee members were female. 

Defining the criteria of job difficulty . Each of the occupational 
committees recommended an Initial set of criteria for rating job difficulty. 
The resolution committee integrated their suggestions Into a single 11st 
which they referred back to the occupational committees. Committee members 
then rated each proposed criterion on a scale of 1-7 according to how 
important they thought 1t was to Include the criterion 1n the new job eval- 
uation system. The resolution committee used the average rating of each 
criterion's importance as a guide 1n deciding which ones to retain for 
further consideration. The retained criteria were referred back to the 
occupational committees so that levels of difficulty could be defined for 
each one (e.g., what kinds of skill at working with people should be 
measured and how difficult are those skills compared to each other?). The 
recommendations were integrated by Dr. Beuhrlng and Mr. Erickson and reviewed 
by the resolution committee for completeness. Members of the occupational 
committees then rank ordered the proposed levels of difficulty from low to 
high (e.g., they rank ordered activities such as "exchanging clear-cut 
Information", "explaining policies and procedures", "mediating disputes", 
and "negotiating contracts" according to how much skill at working with people 
each one required). Since this ranking procedure 1s an Interval scaling 




technique, 1t was appropriate to use the average rank assigned to each 
level of difficulty as Its Initial point value. These point values were 
later standardized to control for differences In the number of levels that 
had been defined for each criterion. The standardized points were then 
weighted according to the results of the employee surveys described below. 

Refinement of t he criteria . Each preliminary criterion, and the levels 
of difficulty that were defined for 1t^ was turned Into a multiple choice 
question. A random sample of employees In 65 different classes of work 
were then asked to use the questions to rate the difficulty of their own 
jobs (e.g., "Which of the following skills at working with people does your 
job require most often? Give an example."). The employees' supervisors 
were asked to review the answers and any disagreements of opinion between 
an employee and his or her supervisor were resolved with the help of the 
Personnel Department. 

Statistical analyses of the answers Indicated which questions were 
useful for measuring differences among related classes of work (e.g., Secretary, 
Senior Secretary, and Principal Secretary) and which questions did so in 
a relatively unbiased manner (e.g., did the question measure differences among 
typically male classes of work better than it measured differences among 
typically female classes of work, or did it measure differences about equally 
well in both cases even when one group scored higher than the other on the 
average?). Only questions that were both useful and relatively unbiased were 
retained. 

A subsampie of employees were asked to complete the questionnaire a 
second time. Statistical analyses Indicated how consistently employees could 
Interpret and answer each question from one time to the next. Only questions 
that could be answered with some consistency were retained. Examples that 
had been given to support each answer were reviewed to see what modifications 
1n wording might be made to further Improve the reliability and validity of 
the questionnaire. 
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The final job evaluation questionnaire . The final questionnaire covers 
37 specific criteria of Job difficulty (1n order to simplify the format, some 
criteria are measured by more than one question). These criteria are 
organized Into clusters that represent nine major aspects of a job's respon- 
sibilities and requirements: the amount of Knowledge, Skill, Mental Effort, 
Physical Effort, Independent Judgment, and Supervisory Responsibility the job 
requires, the amount of Risk 1t Involves, and the Effect of Error and Impact 
of carrying out the job's duties (see Appendix I for details). 

W eighting the criteria . A general survey returned by 1,600 employees 
determined the weight that should be assigned to each of the final nine clusters 
of criteria. A survey returned by 120 coitmittee members determined the weight 
that was assigned to each criterion within a cluster. The weights reflected 
employee judgments about the relative importance each cluster should have 
in determining salaries (or the relative Importance of the criteria within 
in each cluster). The surveys asked for "ratio" judgments of relative 
Importance Instead of a simple rating of each cluster's or criterion's 
importance on Its own. While this made the surveys somewhat more difficult 
to complete, 1t ensured that we could state with accuracy that employees 
believed a particular cluster was twice as important as another cluster, and 
so on. 

Two sets of analyses were conducted to determine whether the weights 
assigned by the surveys truly reflected employee opinions. The weights 
determined from the surveys that were returned by 1,600 of the University's 
8,000 employees turned out to be nearly Identical to the weights that were 
assigned by an Independent sample of employees who were randomly chosen to 
be representative of the entire employee population, when the 1,600 surveys 
were divided into nine major occupational groups according to the jobs held 
by the employees who filled them out, 1t was clear there was consensus across 
the groups about the relative Importance of each cluster of criteria (e.g., 
physical operations employees agreed with clerical employees that Knowledge 
should be neany three times as important as Physical Effort 1n determining 
the salary for a class of work). These analyses Indicated that the general 
survey results fairly represented employee opinions throughout the workforce. 
This was critical since the weights ultimately influenced the points a job 
could receive when it was rated (see Appendix I). 
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Rating the difficulty o f Individual .jobs . As described earlier, the 
final Job evaluation questionnaire can be used by an employee to rate the 
difficulty of his or her own job. To do this, the employee marks the one 
answer to each question that best describes the job's requirements and gives 
an example to support that answer. The employee then gives the completed 
questionnaire to his or her supervisor to review. For the study that determined 
the difficulty of each class of work, the employee and supervisor were asked 
to resolve disagreements among themselves before sending the questionnaire In 
to be scored. For ongoing classification purposes, only the supervisor's 
answers will be scored since the employee has the option of the grieving the 
classification outcome if s/he disagreed with the supervisor's answers. 

Rating the difficulty of classes of work . To determine the average 
difficulty of each class of work, a group of employees who were randomly 
selected from each class were asked to fill out the job evaluation questionnaire. 
The questionnaires were scored and the scores for the jobs within a particu- 
lar class were averaged to determine the difficulty of the class as a whole. 
The range of scores for those jobs was also used as an Indication of how 
much the jobs within a class usually vary in difficulty (this information 
will be used later as a guide for classifying individual jobs). Appendix II 
provides further details on how and why this procedure was used to determine 
the difficulty of each class of work. 

Reliability and validity of the ratings . The confidence you can have 
in a questionnaire is determined by (1) the reliability of its results (how 
likely the job is to get the same rating again) and (2) its validity (whether 
the ratings really measure job difficulty). It was critical to assess the 
questionnaire's reliability because the less consistently it rated job difficulty 
the less valid those ratings would be. It was even more Important to assess 
the questionnaire's validity since it was always possible that it would 
consistently measure something other than job difficulty. 

Reliability was assessed by asking 100 randomly selected employees to 
fill out the final questionnaire twice, a month-and-a-half apart. They were 
not warned that the second rating would be requested so there was no Incentive 
to save a copy of their first rating or to memorize their answers. Their 
supervisors reviewed the answers each time and resolved any disagreements of 
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opinion. A statistical comparison of the two ratings for each job Indicated 
that, while the ratings were never Identical, the total score for the job 
usually remained much the same (r « .96 out of 1.00 possible). This very high 
level of reliability is due partly to the fact that the questionnaire measured 
so many specific aspects of a job's responsibilities. Although changes In 
judgments were possible on any one question, the changes that alone would raise the 
job's score were usually offset by other changes that lowered the job's score. 
The net result was that the total score remained approximately the same. 

The test-retest reliability of .96 compares very favorable with the 
interrater rel1ablit1es of .60 to .85 that have been reported 1n the literature 
for other methods of job evaluation. 

Determining the validity of the new job evaluation system was particularly 
important because the criteria of job difficulty had been chosen and weighted 
by employees rather than by management. Validity was therefore assessed in 
a variety of ways. Each test of the system's validity Involved comparing 
the new system's rating of the difficulty of a class of work with a more 
conventional rating of Its difficulty. 

The most conventional standard of difficulty 1s the current salary grade 
for a class of work. There was a very strong relationship between current 
salary grades and the average ratings of difficulty for 125 classes of work 
(r = .88). The relationship was also strong when the classes were divided 
Into three groups based on the predominant sex of their employees (r ■ .90 
tor male-dominated classes, r * .90 for mixed classes, and r c .84 for female- 
dominated classes; the latter correlation was probably lower because the salary 
structure for those classes has been distorted in recent years by selective 
adjustments that were tied to the Implementation of comparable worth at the 
State). The strength and pattern of these correlations indicate that the 
new job evaluation system measures differences in job difficulty as adequately 
as the current salary structure does for a wide variety of different types 
of work. The correlations compare favorably to the correlations with salary 
of .80 to .88 that have been reported for widely accepted consultant-designed 
systems such as the Arthur Young Decision-Band method and the Hay Associates 
Guide Chart method. 
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Experts In job evaluation within the Personnel Department were Involved 
1n establishing another standard of job difficulty. Fifteen employees with 
considerable experience In job analysis and evaluation were asked to rate 
general descriptions of 60 classes of work on five different dimensions: the 
amount of Knowledge, Skill, Mental Effort, Independent Judgment, and Supervisory 
Responsibility they each required. Their ratings on each dimension were then 
integrated by a statistical technique called multl -dimensional scaling. The 
single scaled rating for each class was then compared to the average difficulty 
ratings that had been obtained using the new job evaluation questionnaire. 
There was a strong relationship between the two ratings (r = .88). It was 
particularly strong given how very different the two rating methods had been. 
The strength of this relationship indicated that the employee criteria of job 
difficulty were compatible with the criteria that the Personnel Department 
has used In the past. 

Two additional estimates of validity were obtained by taking advantage 
of ratings that were made by two widely accepted consultant systems for reasons 
unrelated to this project. The Hay Associates Guide Chart system has been 
used to rate several hundred classes of work at the State of Minnesota: 68 of 
those classes are similar to ones at the University. There was a moderately 
strong relationship (r ■ .83) between Hay ratings of the State's classes and 
the questionnaire-based ratings of similar classes at the University, even 
though the job matches were not always good (in ten cases 1t took two State 
classes to match one at the University, or vice versa). 

The Arthur Young Decision-Band method has been used to rate nearly 200 
health-related classifications at the University Hospital. In a joint study 
with the Hospital, 22 of their classes and 21 University classes were rated 
by both the Decision-Band method and the University's new questionnaire approach. 
The relationship between the two sets of ratings for the 43 classes was 
strong (r_ ■ .87) even though the sample of classes was not representative 
of the general workforce in either location. 

The outcome of the new system's comparison to these two consultant-designed 
systems 1s consistent with the correlations of .85 to .95 that have been 
reported in the literature when different consultant-designed systems are 
conpared to each other on an identical set of jobs. 
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All of the above tests are consistent In Indicating that the employee 
standards of job difficulty are at least as valid as other, more conventional, 
standards. It Is Important to note, however, that high correlations do not 
mean Identical results. Some changes will be necessitated by the new system 
and every effort has been made to ensure that each change 1s valid 1n Its 
own right. 



III. ADJUSTING SALARY GRADES FOR CLASSES OF WORK 

The primary purpose of this study was to determine how salary grades 
should be adjusted 1n order to pay each class of work according to its rated 
difficulty, regardless of the predominant sex of its incumbents. This would 
produce an internally equitable salary structure that could be maintained by 
the University through periodic re-evaluations in future years. A second 
purpose was to comply with a State statute that required the University to 
conduct an analysis of its current salary practices for primarily male versus 
primarily female classes of comparable difficulty. 

Method for including classes in the study . The study was conducted in 
two phases for practical reasons. A total of 125 out of more than 450 non- 
hospital classes of work were tested 1n the first phase to determine the average 
level of their difficulty. These classes included all those with twenty or more 
Incumbents plus a random sample of smaller classes to ensure representative 
results. Of these classes, 52 were female-dominated (70S or more female 
incumbents in the class), 42 were male-dominated (80* or more male incumbents), 
and 31 were mixed or "balanced" (21% to 69% female incumbents). The definitions 
of sex-dominance were dictated by the State statute. Together, the 125 
classes cover three quarters of the University's non-hospital employees. 
The remaining 325 or so classes that cover the remaining one-quarter of the 
employees were tested later 1n Phase II. 

Determining pay inequities . To comply with the State statute, a 
statistical technique called regression was used to determine whether female- 
dominated classes are currently being paid the same as male-dominated classes 
of similar difficulty. In keeping with the outcomes of similar such studies, 
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the results Indicated they are not. On the average, male -dominated classes 
are being paid $2.37 per point of rated job difficulty while female -dominated 
classes are being paid only $1.81 per point. 1 In addition, the amount of 
underpayment Increases as the rated difficulty of the female-dominated class 
Increases. The following 1s an example of the discrepancies: 



Rated Job Difficulty 





1200 points 


1600 points 


2000 points 


male-dominated class 


$1199/mo. 


$2147/mo. 


$3095/mo. 


female-dominated class 


$1147/mo. 


$1871/mo. 


$2595/mo. 



dollar underpayment $ 52/mo. $ 276/mo. $ 500/mo. 

percentage underpayment 4% 13% \$% 



The results for all 125 classes are Illustrated in the first graph of 
Appendix III. Female-dominated classes are plotted as "F", male-dominated 
classes as "M", and balanced classes as "B". The average point total for 
a class can be determined by reading straight down from the plotted letter 
to the bottom of the graph; the midpoint of the current salary range for 
the class can be determined by reading straight across from the plotted 
letter to the left margin of the graph. 

The male and female pay lines on the graph were computed according to 
the formulas Indicated in footnote 1 below. The lines themselves show the 
average current midpoint salary for male-dominated classes at each point 
value, compared to the average current midpoint salary for female- dominated 
classes with the same point rating. The female pay line ranges from 4% to 
16% lower than the male pay line, depending on the difficulty of the class 
(the greater the difficulty, the more the class is underpaid). 2 



The formula for male-dominated classes 1s actually $2.37 minus $1,645; for 
female classes 1t 1s $1.81 per point minus $1,025, 

2 

This 1s probably due to the fact that pay equity adjustments made by the 
State 1n 1983-84 have already been passed on to 20 matching female-dominated 
classes at the University, most of which are lower rated classes of work. 
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Remedying pay Inequities . The principle of pay equity 1s that classes 
of similar rated difficulty should be paid similarly. In order to resolve 
pay Inequities, 1t 1s necessary to determine the appropriate salary range for 
each class according to a common formula. Then existing salary ranges are 
adjusted up or down to the level Indicated by that formula. In practice, this 
means raising or lowering the existing salary ranges for all classes 1n such 
a way that the midpoints of their salary ranges fall on the same pay line, 

The University's general counsel Indicates that the University will be 
vulnerable to legal suits unless this principle 1s applied to all classes 
regardless of the sex of their Incumbents. In addition to being legally 
appropriate, consistent application of the principle ensures that the 
structural relationship among related classes will be equitable regardless 
of whether the classes are female -dominated, balanced, or male -dominated. 

The male pay line formula was used as the common formula for all classes 
primarily because the salary ranges for overpaid classes can usually be 
lowered to the male pay line without making them uncompetetive 1n the 
marketplace. In the few cases when the new lower salary range 1s uncompeti- 
tive and produces documented recruiting problems, a market-related adjustment 
will be made (such adjustments will be reviewed annually to determine whether 
they continue to be appropriate). 

The result of bringing all 125 classes 1n Phase I to the male pay Une 
is Illustrated in the second graph of Appendix III. Note that despite the 
average results, not every female -dominated class 1s underpaid compared to 
male-dominated classes of similar difficulty. Conversely, male-dominated 
classes of similar difficulty are not always paid the same. As a result, 
some male -dominated classes will receive pay equity increases while a few 
female-dominated classes will not. 

Effect on individual salaries . The salaries for individuals 1n an under- 
paid class will be raised by the same percentage that the midpoint of the 
salary range for the class 1s raised. Employees in an overpaid class who 
are paid within the new lower salary range for the class will continue to 
receive regular salary adjustments until they reach the new range maximum. 
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The salaries for Individuals 1n an overpaid class who are at or above the 
new range maximum already will be frozen until their salary falls within 
the new range (lump sum payments that don't affect an Individual's base 
salary may be possible 1n the meantime, however). Individuals 1n classes 
that receive market adjustments will be paid within the market-adjusted 
salary range. New employees will be hired at a salary that 1s within the 
new higher or lower range for their class (unless 1t 1s market-adjusted). 

Implementation. The total cost for raising the salaries of employees 
In underpaid classes is $12.7 million. This 1s approximately 6% of base 
payroll for all non-hospital civil service employees at the University. 
The cost is $8.7 million for the female-dominated classes alone, which 1s 
approximately 4% of base payroll. 

Implementation of the pay equity adjustments will take place over a minimum 
of six years, although the maximum timetable could be considerably longer 
if the State chooses not to help fund the solution and civil service pay 
plans are at or below the cost-of-living for several years. 



IV. USING THE SYSTEM TO CLASSIFY JOBS 

Beginning July 1, 1985 the system will be installed as a more objective 
method for classifying new jobs and for reclassifying existing ones. The 
information obtained during the pay equity study will serve as a guide. 

Defining a point range for each class of work . S- ice a large number of 
jobs in each class have already been rated, it's possible to use the range 
of their scores as an indication of how much the jobs in a class usually 
vary in difficulty. In order to make the range useful for classifying jobs 
in the future, the ranges of scores that were typical for related classes 
(e.g., Secretary, Senior Secretary, Principal Secretary, Executive Secretary) 
were restricted in a way that eliminated any overlap between them. For exampl 
the range for Senior Secretary Is 1251 - 1400 points and the range for 
Principal Secretary is 1401 - 1525 points. If the actual ranges of points 
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overlapped almost entirely, the classes will be combined Instead (e.g., 
Secretary and Senior Secretary will eventually be combined Into a single 
class with a range of 1?51 - 1400 points). 

This method of defining point ranges recognizes that Jobs within a class 
will always vary somewhat 1n difficulty while keeping the amount of that 
variation within reasonable limits. 

Classifying jobs . In order to classify a Job, 1t 1s first necessary 
to review Its duties to see which set of related classes 1s most appropriate 
(e.g., the secretarial set of classes, the word processing set, the account 
clerk set, etc.). The job 1s then assigned to a particular class within 
the set according to Its rated difficulty. 

If It's a new job, the supervisor fills out the questionnaire that is 
used to rate Its difficulty. If It's a request for reclassification Into 
a higher class, or a departmental survey of an existing job, then both the 
supervisor and the employee fill out the questionnaire (although only the 
supervisor's answers are scored). The Personnel Department then reviews the 
answers and supporting examples, resolves with the supervisor any apparent 
errors in interpretation, and sends the questionnaire to be scored. 

The score for the job 1s then compared to the range of scores that 
has been defined for each class 1n the appropriate set of classes. The 
job is assigned to the class that includes other jobs of similar difficulty. 
For example, 1f the Senior Secretary class has a range of 1251 - 1400 points, 
the Principal Secretary class has a range of 1401 - 1525 points, and a new 
secretarial job has a score of 1478 points, then the new job will be 
assigned to the Principal Secretary class. 

This method provides a more objective basis for making difficult 
classification decisions and for resolving grievances about such decisions. 

15 
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APPENDIX I 



University of Minnesota 
Civil Service Job Evaluation System 



ITEMS AND CLUSTERS 



I. KNOWLEDGE (16% of points) 

1. General knowledge 

2. Project-specific knowledge 

3. Knowledge of non-University organizations 

4. Continuing education 

5. On-the-job experience and training 



II. SKILL (15% of points) 

6. Skill at working with data 

7. Skill at working with machines, plants, or animals 

8. Skill at working with people 

9. Skill at working with people using technical terms 
(or foreign language) 

10. Skill at writing standard English 

11. Skill at writing with technical terms (or foreign language) 



III. MENTAL EFFORT (13% of points) 

12. Complexity of duties 

13. Initiating or planning 

14. Problem solving 

IV. INDEPENDENT JUDGMENT (13% of points) 

15. Non-supervisory guidelines 

16. Supervisory guidelines 

V. SUPERVISORY RESPONSIBILITY (10% of points) 

17. Formal supervisory responsibility 

18. Lead worker responsibility 

19. Responsibility for training, evaluating, reviewing, or 
directing work 

20. Number supervised (in ways described above) 

21. Time spent supervising (in ways described above) 
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APPENDIX I, Continued, 



VI. EFFEC7 OF ERROR (10% of points) 

22. 1n working with people 

23. 1n working with data 

24. 1r. working with machines, plants, or animals 

25. 1n solving problems 

26. 1n supervising (In ways described above) 



VII. IMPACT OF DUTIES (10% of points) 

27. Impact of working with people 

28. Impact of working with data 

29. Impact of working with machines, plants, or animals 

30. Impact of problem solving 

31. Impact of supervisory responsibility 



VIII. RISK (6i% of points) 

32. Risk to employee (health or accident) 

33. Time exposed to risk 

34. Responsibility for protecting others 

35. Risk to those protected 

36. Number protected (directly vs. indirectly) 



IX. PHYSICAL EFFORT (6i% of points) 
37. Straining the body or senses 
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APPENDIX II 

University of Minnesota 
Civil Service Job Evaluation System 



THE RATING OF JOBS AND CLASSIFICATIONS 

The new job evaluation system will be used to determine which classes with dif- 
ferent duties are of equal difficulty and so should be paid equally* and which 
classes are of greater or lesser difficulty by comparison. 

Many people argue this is like trying to compare apples and oranges. They're 
right. Apples and oranges can be compared, however, on some basic criteria that 
are common to them both: number of calories, number of vitamins, amount of fiber 
content. Rating apples and oranges on these common criteria permits you to com- 
pare these fruits based on their overall nutritional value. The greater the 
nutritional value, the higher the price you should be willing to pay. 

In a similar way, rating classifications on a common set of criteria permits 
you to compare the classes on the basis of their overall difficulty. In this 
cas<», the greater their difficulty, the more the University should be willing 
to pay. University employees made the value judgments about which criteria 
should be used to rate job difficulty and how heavily those criteria should be 
weighted. The next step was to use these criteria to rate the difficulty of the 
University's job classifications. That process is described below. 

Rating individual jobs 

The 37 factors in the final job evaluation system were turned into a multiple- 
choice questionnaire. Employees used this questionnaire to rate the difficulty 
of their own jobs. To do this, they marked the one answer to each question 
that best described their job's requirements. * The employee then reviewed the 
answers with his or her supervisor before returning the questionnaire for 
scoring. The scorinq system assigned points to each answer according to its 
level of difficulty.? The points for each answer were then added together. 
This point total indicated the overall difficulty of the job. 

Job evaluation vs. performance evaluation 

The point total for a job rates only the difficulty of the job's duties. It 
does not rate how well those duties are performed. Although both types of 
rating are important, they must be handled by different systems because they 



1 A test of the questionnaire indicated that this approach measured differences 
in job difficulty better than the method of allowing employees to mark every 
answer that applied or the method of allowing them to check the most dif- 
ficult level that was ever done on their job. 

2 Over 120 commi ttee members rank ordered the answers for each questi on 
according to their level of difficulty. The average rank order of each 
answer was used as its initial point value. This initial point value was 
then standardized and multiplied by the weight assigned to the question in 
the employee weighting surveys. 
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APPENDIX II, Continued 

serve different purposes. Job evaluation rates the job: It's used to set salary 
ranges according to the difficulty of the job's duties, regardless of who does 
them. Performance evaluation rates the person: It's used to determine merit 
raises for Individuals based on how well they perform their duties. Job evalu- 
ation Is Important 1n a more general sense because 1t sets the salary range 
within which those merit raises can be given. 

Rating classifications 

The main purpose of job evaluation 1s to assign classifications to salary ranges 
according to the average difficulty of their duties and responsibilities. Since 
individual jobs within a class are seldom equally difficult, and since generic 
descriptions of class responsibilities are difficult to write accurately and 
don't reflect this variation among jobs, we decided to rate several randomly 
selected jobs in each class and average their ratings to get the point total for 
the class as a whole. 3 

An example 

The following is an example of how some real jobs in two different classes were 
rated. Although only a few ratings are shown in these examples, we actually 
rated 10-25 jobs in every class with more than twenty employees and most or all 
of the jobs in smaller classes. 



Job Evaluation 
Clusters 



Knowledge 
Skill 

Mental Effort 

Indep. Judgment 

Supervision 

Error 

Impact 

Physical Effort 
Risk 

Job point total 



Class average: 



One Class 



Job A 


Job B Job C 


201 


201 


214 


206 


220 


233 


141 


179 


182 


146 


173 


184 


112 


112 


122 


106 


109 


131 


125 


122 


143 


87 


99 


94 


84 


85 


88 


1208 
-\ 


1300 


1391 




-v 




1300 


total 


points 



Another Class 



Job A Job B Job C 



1203 1300 1397 



1300 total points 



3 The median, rather than the arithmetic mean, was used as the average score. 
The median is the middle score in a set of scores that have been arranged from 
low to hfph. For example, the scores 1000, 1020, 1040, 1060 and 1880 have a 
median of 1040 versus an arithmetic average of 1200 points. As can be seen, 
the median is much less vulnerable to the very extreme scores that are likely 
to occur here due to the misclassif ication of jobs and the tendency of some 
employees to underrate or exaggerate their job's requirements. 
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APPENDIX II, Continued 



This example shows two Important things. First, Individual Jobs within a class 
may get somewhat different point totals even though they score high and low on 
the same clusters of items. In other words, even though jobs within a class 
have the same pattern of responsibilities, some jobs may be more difficult 
overall because they require a bit more of each type of responsibility. 

Second, two classes may get the same average point total even though their jobs 
score high on different clusters of items. That is, although each class gets 
credit for its unique pattern of responsibilities, their average point totals 
may show that the classes are equally difficult overall and so should be paid 
equally. Some classes, of course, will be rated as more difficult than others 
because they score high on more clusters. Just how much more difficult they 
are, and consequently how much more they should be paid, will be shown by the 
size of their average point totals. 

The main thing to remember is that the average point total for a class is de- 
termined by ratings of individual jobs that were made by employees who do the 
work. Thus it is employee judgments that ultimately determine how difficult 
each class is and how much it deserves to be paid. 
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